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ABSTRACT 

In this paper, we consider orthogonal matching pursuit (OMP) al- 
gorithm for multiple measurement vectors (MMV) problem. The 
robustness of OMPMMV is studied under general perturbations — 
when the measurement vectors as well as the sensing matrix are in- 
corporated with additive noise. The main result shows that although 
exact recovery of the sparse solutions is unrealistic in noisy scenario, 
recovery of the support set of the solutions is guaranteed under suit- 
able conditions. Specifically, a sufficient condition is derived that 
guarantees exact recovery of the sparse solutions in noiseless sce- 
nario. 

Index Terms — Multiple measurement vectors (MMV), orthog- 
onal matching pursuit (OMP), compressive sensing (CS), general 
perturbations. 

1. INTRODUCTION 

Finding the sparse solution to an under-determined linear equation 
is one of the most basic problems in some fields of signal process- 
ing. It has also received significant attention since the introduction 
of compressive sensing (CS), which is widely investigated in the past 
decade [Q. The basic mathematical formula is 

y = &x. (1) 

In the field of CS, y G C m denotes the measurement vector, <J> G 
C mx " with m < n is called the sensing matrix, and x G C" is the 
sparse signal to be recovered, which means most of its entries are 
zero. 10 is also termed single measurement vector (SMV) problem. 
Many algorithms, including orthogonal matching pursuit (OMP), are 
proposed to solve the SMV problem. The recovery performances 
of these algorithms are studied in several scenarios, depending on 
whether the noise on y or <I> exists. Many researches have been 
done on the recovery accuracy of OMP in those scenarios 12H41. 

In this paper, the recovery performance of computing sparse so- 
lutions to multiple measurement vectors (MMV) is analyzed. The 
MMV problem was initially motivated by a neuromagnetic inverse 
problem which is involved in Magnetoencephalography [5]. In a re- 
cently proposed system named the modulated wideband converter 
(MWC) (6], MMV recovery also plays a important role in detecting 
the locations of narrowband signals. The MMV problem is formu- 
lated as the following equations 

Y = (2) 
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where Y G C mxi , * G C mxn , and X G C nxL . The matrix Y 
is made up of L measurement vectors, and matrix X is the sparse 
signal to be recovered. Here, a matrix X is called jointly fc-sparse 
or fc-sparse if it contains no more than k nonzero rows. 

Recovery algorithms to the MMV problem include convex re- 
laxation and OMPMMV, which is an extension of OMP to the MMV 
(7JIU- Other algorithms, including FOCal Underdetermined Sys- 
tem Solver (FOCUSS) | 5| and Joint £2,0 Approximation Algorithm 
(JLZA) [9], are also put forward. When Y and <1? are unperturbed, 
the recovery process of OMPMMV can be written as: 

(N ) X = R(Y,$,k), (3) 

where R denotes the recovery process, k denotes the sparsity level, 
and X is the approximation of the original sparse signal X. (No) 
process is an ideal one. It has been shown that under certain matrix- 
norm-based conditions, sparse signal can be exactly recovered (8), 
i.e. X = X. 

More generally, the perturbed observation matrix Y and sensing 
matrix <I> in the form of 

Y = Y + B, * = * + E. (4) 

needs to be taken into account. In such scenario, the recovery pro- 
cess becomes 

(N 2 ) X = R(Y,$,k). (5) 

Such consideration is necessary because perturbations on Y and 
always exist in practice. In many CS scenarios, such as when 3? 
represents a system model J6j, E denotes the system perturbation 
in realization. Also, B denotes the measurement perturbation when 
quantization effects introduce considerable noise to Y . 

As far as we know, few researches have been done yet on the 
robustness of OMPMMV in (N2) process. In this paper, the cor- 
responding theorem for SMV in (4) is extended to MMV under the 
general (N2) scenario. The main result shows that under certain 
conditions based on the Restricted Isometry Property (RIP), the lo- 
cations of nonzero rows of X can be exactly recovered, and an upper 
bound on the recovery error is given. In addition, it is demonstrated 
that such conditions guarantee exact recovery of X in (No) process. 

1.1. Notations and Assumptions 

Assume U G C" x L is a matrix, then its columns and rows are rep- 
resented as Uj, j = 1, . . . , L and U(i), i = 1, . . . , n, respectively. 
Notice that in this paper, upper-case letters are used to denote matri- 
ces. The support set supp(u) denotes the indices of nonzero entries 
in vector u. The support set of a matrix U is defined as supp([7) = 
Uj=i l su PP( u j)- Tne number of elements in supp(n) and in 
supp([/) are denoted as ||u||o and ||l7||o, respectively. 



Table 1. The OMPMMV Algorithm 
Input: Y, *, k; 

Initial state: R° = Y, A = ,1 = 0; 
Repeat 

1 = 1 + 1; 
match step: 

H l = # T H 1-1 ; 
identify step: 

A ! =A ! - 1 U{argnia Xj ||H' ! (j)||2}; 
update step: 

AT' = argmin^ ;sllpp(z) cAi 11^ - ®Z\\ F ; 

R l = Y - &X l ; 
Until Z = k; 
Output: X fc . 



The symbols <T ma x(-)i c m i n (-), ||* ||a< and || ■ ||f, denote the max- 
imum, minimum nonzero singular values, spectral norm, and Frobe- 
nius norm of a matrix, respectively. Let || • || 2 denote the largest 
spectral norm taken over all fc-column submatrices. The perturba- 
tions E and B can be quantified with the following relative bounds, 



1*1 



IB 



— < c 7 — 1 k 



|(0 



< £6 



(6) 



In this paper, it is assumed that eo, e, and Eb are far less than 1. 



2. BACKGROUND 

2.1. Orthogonal Matching Pursuit for MMV (OMPMMV) 

The key idea of OMPMMV, which is similar to OMP, lies in the 
attempt to reconstruct the support set A of X iteratively by starting 
with A = 0. The pseudo-code of OMPMMV is described in Table[T] 
In fact, the process of OMPMMV can be mathematically expressed 
as follows. 

Suppose A C {1, . . . , n}. Let u| A denote the |A| x 1 vector 
containing the entries of u indexed by A. Let *a denote the m x 
|A| matrix obtained by selecting the columns of sensing matrix * 



indexed by A. 



V A 



= (* a *a) X *a denotes the Moore-Penrose 



pseudoinverse of *a- Define Pa = *a*a ant ^ -^a = I -Pa 
as the orthogonal projection operator onto the column space of *a 
and its orthogonal complement, respectively. Define A a = -Pa 1 *- 
From the theory of linear algebra, orthogonal projection operator P 
obeys P = P T = P 2 and that the columns of Aa indexed by A 
equal zero. 

Now, suppose that OMPMMV performs at Zfh iteration, and 
A !_1 is the estimation of supp(X) from the previous iteration. The 
following discussion demonstrates the generation of A . 

In the update step of the previous iteration, which is actually 
solving a least squares problem, it can be derived that 



R 1 = Y-& A i- 1 X l - 

= (i-p A ,_i)y 

In the matching step, one has 



| A ,-i =l r -* A i-i*i,_ 1 l r 



A'- 



-bX = A Al -iX 
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^{P^fP^Y = A A ,_iA Al -iX. 



(7) 



(8) 



Then, in the identify step, A 1 = A i_1 |J{argmaxj \\H l (j)\\ 2 }. 

From Q, l[8), and the fact that the columns of Aa indexed by A 
equal zero, several useful conclusions can be derived. 

1. H l (j) = 0, Vj G A'" 1 . Therefore argmaxj ||H"'(j)l|2 $ 
A'" 1 , IA'- 1 ! =1-1. 

2. l[8j can be written as H l = A^nA^-iX*'^ 1 , where 



X* 



I A' 



-1=0 and X* 



(A 



-1)0 = X|( A I-l)c 



It is easy to check that 1) still holds when Y and * in the above 
analysis are contaminated ones, i.e. 



H l (j) = o, VjeA'- 1 . 



2.2. The Restricted Isometry Property (RIP) 



(9) 



For each integer k = 1, . . . , n, the RIP for any matrix A £ c mxn 
defines the restricted isometry constant (RIC) Sk which is the small- 
est nonnegative number such that 



(l-fc)IMIa < \\Auf 2 < (1 + S k )\\uf 2 



(10) 



holds for any fc-sparse vector u. In other words, A acts as an ap- 
proximate isometry on the set of fc-sparse vectors for a small Sk- 

3. CONTRIBUTIONS 

Theorem 1: Suppose that the inputs Y and * of OMPMMV al- 
gorithm are contaminated by noise as in Define the relative per- 
turbations so, e, and ei as in (|6j. Let to = niin jesupp (x) ||X(j)||2 
and 



eh 



9(2 + e)e 
: 12-8(l + e 



1*1 



+ (eo + ei, + eo£b)||*]|2||y]|F. (11) 
If * satisfies the RIP of order k + 1 with isometry constant 

Jfc+i <Q(k,t /e h ), (12) 

then for any fc-sparse signal X, OMPMMV will recover the support 
set of X exactly from Y and * in k iterations, and the error between 
X and the recovered signal X can be bounded as 



|| X - X||f/||X|| f < (e + e b )F(l/Vk). 
The functions in l !12t and d 1 3 1 > are defined as follows: 
1 1 



(13) 



Q(u,v) 



2^/Ti+l 2^+1 



w, v £ 



F(w) 



1+w 



2-(l + ™)(l + e) 2 



to G (0,1). 



Remark 1: The above conclusion generalizes the result of The- 
orem 1 in |4). If L — 1, then the observation matrix and the sig- 
nal matrix are reduced to vectors, which becomes an SMV problem. 
Thus Theorem 1 here can be regarded as an extension from SMV to 
MMV framework. Notice that the requirements in the above Theo- 
rem 1 and the correspondent theorem in jftj share many similarities 
in form. The differences include the definition of ej, to, and the 
change from £2 norm of y to Frobenius norm of Y . Besides, the 
estimation error of X is derived in a relative form. 



Remark 2: For OMPMMV in the perturbed scenario it is un- 
realistic to achieve the exact recovery of X, because the least square 
problem is actually solved with perturbed data. However, one can 
still recover the support of X. This is of great significance in many 
practical applications, e.g. the MWC |6| mentioned before. 

Remark 3: An MMV problem can be considered as how to 
achieve sparse representations for SMVs simultaneously. However, 
by looking at ej,, we show that MMV has an advantage over a simple 
combination of SMVs. In {6}, Eb is defined as an upper bound on: 



|b||f/II*1f 



EiNil/Eiiwi 



(14) 



3=1 



3=1 



From SMV view, max J= i.... .z,{||foj 1 1 2 / 1 1 3/j 'H2} cannot be too large 
for support recovery. Under the MMV scenario, however, a very 
large ||6j||a/||l/i||a may be balanced by other [|b<||2/lll/ilU> i j, 
and therefore the support recovery is not influenced. 

Though a completely perturbed situation is considered in Theo- 
rem 1, it is helpful to consider three specific situations. The follow- 
ing three corollaries are derived from Theorem 1 . 

Corollary 1: Suppose that 3> satisfies the RIP of order k + 1 
with isometry constant 



Sk+i < 



2Vk + l'' 



(15) 



then OMPMMV will recover X exactly from Y and <1? in k itera- 
tions. 

The proof of Corollary 1 is directly derived by setting B = 
0, E = 0, and £0 = £ = £t — 0. Thus Eh = 0, and i 1 2t reduces to 

03. 

Because SMV is a special case of MMV when L = 1 holds, 
Corollary 1 generalizes the results of Davenport and Liu 1101 . 
It was shown in the SMV case that 8 k +i < l/(3vfe) is sufficient 
for OMP to exactly recover any fc-sparse signal (3) (Th.3.1). Later, 
Liu and Temlyakov relaxed the bound on the isometry constant to 
1/((1 + V2)Vk) ED (Th.5.2). 

Corollary 2: Suppose that Y, <!>, to, and Eb meet the assump- 
tions made in Theorem 1, and <I> = which means that only the 
observation matrix is perturbed. Define 

e hl =e(,||#|| 2 ||l r ||F. (16) 
If $ satisfies the RIP of order k + 1 with isometry constant 

S k+1 < Q(k,t /e hl ), (17) 

then OMPMMV will recover the support set of X exactly from Y 
and 3> in k iterations. The recovery error can be bounded as 



\\X - X|| F /||X|| F < e b (Vk + l)/Vk=T. 



(18) 



Corollary 3: Suppose that Y, to, Eo, and s meet the as- 
sumptions made in Theorem 1, and Y = Y, which means that only 
the sensing matrix is perturbed. Define 

°*= n%*y> (n*ll* + Ill«ll») **hn 

+ £ ||*|| 2 ||11f. (19) 
If $ satisfies the RIP of order k + 1 with isometry constant 

5 k+l < Q(k,t /e h2 ), (20) 



then OMPMMV will recover the support set of X exactly from Y 
and ^ ink iterations. The recovery error can be bounded as 



|X-X||f/||X]| f < eF(l/Vk). 



(21) 



The above two corollaries suggest that the relative recovery error 
scales almost linearly with the noise level Eb or e when k is fixed. 



4. PROOFS 



4.1. Some Lemmas 



Before proceeding to the proof of Theorem 1 , some helpful lemmas 
are provided first. 

Lemma 1: Suppose vector a, b,c G R L , then ||a+b+c||2 < 

1 1 a 1 1 2 — t— 1 1 1> 1 1 2 -1— 1 1 <: 1 1 2 , i.e. for arbitrary a, , h , a £l, i — 1, . . . , L, 



\ 3=1 



\ 3=1 \ 3=1 \ 3=1 



Lemma 2 ( j3j, Lemma 3.1 ): Let u,v £ 1" be given, 
and suppose that matrix $ satisfies the RIP of order max(||it — 
v||o,||u + «||o) with isometry constant 5. Then 

G?v) — {u, v)\ < <5 1 1 -ti 1 1 2 1 1 f 1 1 2 - 

Lemma 3 ( (3)> Lemma 3.2 ): Suppose that * satisfies the 
RIP of order k with isometry constant 8, and let A C {1, . . . , n}. If 
A| < k, then 



|u||| < Pau||! < (i + j)||u||3 



holds for all u e R™ such that ||u||o < k — |A| and supp(u) n A = 
0. 

Lemma 4: Let A C {1, . . . , n} and suppose X* G R" xL 
with supp(X*) n A = 0. Define H = A T A A A X*. Then if * 
satisfies the RIP of order ||X*||o + |A| + 1 with isometry constant 
S, it holds that 



\H(j)-X*(j)\\ 2 < 



\X* 



(22) 



1-5 
for all j £ A. 

Proof. According to the definition of H, it can be implied that 

H(j) = (h 1 (j),...,h L (j)) 

= ({A A x*,A A ej),- ■ ■ , (A A x* L ,A A ej)), (23) 

where e.j denotes the j-th vector from the cardinal basis. Lemma 3 
implies that the restriction of A A to the columns indexed by A c sat- 
isfies the RIP of order {\\X* || + |A| + 1) - |A| = \\X* \\a + 1 with 
isometry constant 5/(1 — 5). 

Consider the scenario of j A. Because for alii 6 {1, ... , L}, 
||a5i±ej||o< ||Xi||o+l< ||X*||o+landsupp(Xi±ej)nA = 0, 
it can be concluded from Lemma 2 that 



l^i(i) - = \{A A x* , A A ej) - {x*,ej)\ 

\\<B*ih\\ e ih 



< 



1-5' 
5 



\\ x l 2, 



(24) 



Combining ( 124b with d23l l. it can be easily derived that 

0-' 

-l|(||x*||a,...,l|"£||a)||a 



H(j) - X\f)\\ % = \\(hi(j) - xt(j), h L (j) - xl(j))\\ 2 
8 



1-4 

-J-\ 
1-4 



X* 



□ 



Lemma 5: Suppose that A, <1>, X* , H, 8 meet the assump- 
tions specified in Lemma 4, C is a constant, and H is the matrix in 
the matching step of OMPMMV which satisfies \\H — H ||p < C. 
If 

max \\X*(j)\\ 2 > ^L\\X*\\v+2C, (25) 
?e{i, ...,»} 1 — d 

it is guaranteed that argmaxj ||.ff(j)||2 G supp(X*). 

Proof. According to Lemma 4, for all j £ A, 1 1 H ( j ) — X * ( j ) 1 1 2 < 
<5||X*|| P /(l-<5),thus 

\\H(j) - X*(j)\\ 2 < \\H(j) - H(j)\\ 2 + \\H(j) - X*(j)|| 2 

< \\H-H\\ F + \\H(j)-X*(j)h 

„ 5 



1 - 5 



\X*\\ F + C. 



(26) 



Then for indices j ^ supp(X*), we will have ||.ff(j)||2 < 
<5]|X*]|f/(1 - 5) + C (Recall that H(j) = for j G A). 
If d25t is satisfied, then there exists some j G supp(X*) with 
||X*(j)|| 2 > 25||X*||f/(1 - 5) + 2C. From ^ and the 
triangle inequality, it can be concluded that for this index j, 
\\H(j)\\ 2 >6\\X*\\ F /(l-6) + C. □ 

Lemma 6: Suppose that \\H l — H l || F < C holds in the Z-th 
(I — 1, . . . , k) iteration during OMPMMV running, and <& satisfies 
the RIP of order k + 1 with isometry constant 



4+i < Q(k,t /C). 



(27) 



Here, C and to = niinj esupp (x) ||-^C?')||2 are two constants. Then 
OMPMMV will recover the support set of X exactly from Y and # 
in k iterations. 

Proof. The proof works by induction. To begin with, consider the 
first iteration where A = 0. Because to < || X ||f, one has 

<5||x|| +i < 4+i < Q(k,t /C) < Q(k, \\X\\ F /C), 
and this implies that 



Vk 



> 2< V"° +1 \\xy + 2C. 



1-6 



l|X||o+l 



Notice that max^j! „j ||Jf*(j')|| 2 is no less than \\X \\-p/Vk. 

Therefore J25l > can be satisfied. Consequently, according to Lemma 5, 
it is proved that 

argmax || H (j)\\ 2 G supp(X). 

j 

Furthermore, consider the general induction step. Suppose that 
OMPMMV is at Zfh iteration and all previous iterations succeed, 
which means that A' -1 is a subset of supp(X). Recall that the in- 
tersection of supp(X* !_1 ) and A' -1 is empty, jA !_1 | equals I — 1, 



and || X* || o is not greater than k — I + 1. Assume that <1? satisfies 
+ |A !_1 | + 1 with isometry constant 



the RIP of order ||X*' 



Because 



|Jf * z_1 ||o + IA'" 1 ! + 1 < + 1, 



and from the fact that to is not greater than ||X* ||f, one has 

S 1 - 1 < 4+i < Q(k,t /C) < Q(k, WX'^Wf/C), 
which implies that 



(28) 



\/k 



25 

— > 



1 - 6 1 - 1 



|X* ! - 1 || F + 2C. 



Notice that 



II X* 1 - 1 ' 
max \\X*(j)\\ 2 > ^— 
je{i,...,n} u ~ Vfc-Z + 1 ~ Vk 



? > ll-X"* 



(29) 



Thus i25i can be satisfied. Therefore, according to Lemma 5, it can 
be concluded that 

arg max || H l (j) || 2 G supp(X* i_1 ), 



and thus 



A C supp(X), 
which completes the proof of induction. 



4.2. Proof of Theorem 1 



□ 



Proof. Similar to the scenario of OMP, here we need to give the 
upper bound on \\H l - H l \\ v (for all / G {1, . . . , k}), and then 
replace the C in Lemma 6 with this bound. AH 1 = H l — H 1 can 
be written as AH 1 = (Ah[, . . . , Ah l L ), where 

Ah\ = hi 1 - h\ =E T (I - P A i-i) Vl - * T (P A i-i - P A i-i)Vi 
+ * T (I - P A i-i)(Vi - Vi), i = 1, . . . , L. 

From the proof of (38) in |4), there is 

\\Ah\h <l|^ T ||2||i--P A i-i||2||yi||2 

+ ||* T || 2 ||P Ai - 1 -P A ,_! [| a || w || a 
+ ||* T || 2 ||I-P A! -i|| 2 ||y I -?/ I || 2 
. 9(2 + e)e 



12-8(l+e) 2 V 11 " 3 H 



l*l|2||tt|| 2 



+ eo||*|| 2 ||yi||2 + (so + l)||*|| 2 ||6i||2 
Combining this and Lemma 1, one gets 



\AH l h=, 



9(2 + e)e 



12 



+ eo||#||2 



l*l|a + f 11*112 



1*1 



\ 3=1 



» Eiiy J n 2 + ( e « + 1 )ii*i 



\ 3=1 



2 

■j 1 1 2 - 



(30) 



Notice that 
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\ 3=1 



\| 3=1 



3lll = ||B||F<e i ||l r || F . (3D 



Applying ( 1311 ) to BOb . one gets the upper bound on || Ai? ! ||f: 



\\AH'h<- 



9(2 + e)e 



12-8(l + e) 2 

+ (so+S6+Soe6)||*||2||y]|F. 

At the end of the proof, J 1 3t is proved as follows. According to 
$2% and {4]l, there is 

y = 4?x £X + s. 

Because A = supp(X) is exactly recovered, one has 



X\ A = &{Y = X\ A + & A (-EX + B). 



Thus 



\X-X\ 

IXIIf 



\E 



( fc )i 



X\\f+\\B\\ 



1 



\E 



+ £&- 



||g||W +£b ||y|| F /||x|| F 

v/2-(l + (5 fc )(H-e) 2 ' 



XI 



Notice that 

l|X|| F ||X||f 
it can be concluded that 

I|X-X||f 



IXI 



< vT+J k 



XI 



< (e + e 6 ) 



2_ (1 +t)(l + £ )^ (£ + gb)F(4) - 

(32) 

It has been mentioned in (5} that the theoretical upper bound of 5fe+i 
is for exact recovery of support set, if <5fc+i is used as a suf- 

ficient condition for recovery of x. Thus, 5k < <5fc+i < 1/vfc. 
Applying this to <32t . one finally gets 1113) . 

□ 
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5. CONCLUSION 

In this paper, robustness of OMPMMV under general perturbations, 
which are in the form of Y = Y + B and $ = $ + E, is stud- 
ied. Though exact recovery of the sparse solutions X from Y and 
<{> is no longer realistic, Theorem 1 shows that exact recovery of 
the support set can be guaranteed under suitable conditions, which 
is important in many practical applications. Furthermore, the recov- 
ery error is bounded. This completely perturbed framework extends 
some prior work in t3ll4l[70l from SMV to MMV. 



